Daichi's Painting: Brushing Up Painting Skills with BrushDevice

We realize a painting system that imitates painting on real objects in the real world. Many studies have proposed methods of painting simulation on 2D canvas, and the quality of such methods is improving. However, painting on 3D objects is not easy using conventional input devices such as a mouse or graphics tablet with 2D display. We solved this problem by providing a ToolDevice in a mixed reality (MR) space. Our device, called BrushDevice, imitates paint brushes; users can change the brush type and stroke weight to express their creativity. INTRODUCTION There have been many studies on painting simulation and their quality is improving [1][2]. In these studies, simulated painting using oils and watercolors on 2D canvas has been realized using conventional input devices such as a mouse or graphics tablet with 2D display. On the other hand, in the real world, we paint on not only 2D canvases but also 3D objects. However, painting on 3D objects has not been considered in these studies. It is difficult to paint on such targets indirectly using a mouse or graphics tablet, and it is natural for users to want to paint the target directly. Therefore, we realized a painting system that imitates painting on real objects in the real world. This study considers the following story: Daichi is a boy with good painting skills. He paints pictures on furniture, pottery, and stationery everyday. He has also attempted computer drawing, but has no sense of computers. One day, he accepted a present from his friend with the message “Express your creative mind using this BrushDevice with a computer!!” He started painting spontaneously and avidly using this device. To realize this story, we first developed a system in which users can directly paint on various objects in MR space and then developed BrushDevice for painting on 2D/3D objects, in a similar manner to how they would in the real world (Figure 1). A system for painting on virtual 3D objects using PHANTOM has already been proposed by Sandor et al. [3], however, it does not target real objects. The targets in our study were both real and virtual objects (Figure 2).